Verbal analogy problem sets: An inventory of testing materials.

Analogical reasoning is an active topic of investigation across education, artificial intelligence (AI), cognitive psychology, and related fields. In all fields of inquiry, explicit analogy problems provide useful tools for investigating the mechanisms underlying analogical reasoning. Such sets have been developed by researchers working in the fields of educational testing, AI, and cognitive psychology. However, these analogy tests have not been systematically made accessible across all the relevant fields. The present paper aims to remedy this situation by presenting a working inventory of verbal analogy problem sets, intended to capture and organize sets from diverse sources

Large Language Model Displays Emergent Ability to Interpret Novel Literary Metaphors

Recent advances in the performance of large language models (LLMs) have sparked debate over whether, given sufficient training, high-level human abilities emerge in such generic forms of artificial intelligence (AI). Despite the exceptional performance of LLMs on a wide range of tasks involving natural language processing and reasoning, there has been sharp disagreement as to whether their abilities extend to more creative human abilities. A core example is the ability to interpret novel metaphors. Given the enormous and non curated text corpora used to train LLMs, a serious obstacle to designing tests is the requirement of finding novel yet high quality metaphors that are unlikely to have been included in the training data. Here we assessed the ability of GPT4, a state of the art large language model, to provide natural-language interpretations of novel literary metaphors drawn from Serbian poetry and translated into English. Despite exhibiting no signs of having been exposed to these metaphors previously, the AI system consistently produced detailed and incisive interpretations. Human judges, blind to the fact that an AI model was involved, rated metaphor interpretations generated by GPT4 as superior to those provided by a group of college students. In interpreting reversed metaphors, GPT4, as well as humans, exhibited signs of sensitivity to the Gricean cooperative principle. In addition, for several novel English poems GPT4 produced interpretations that were rated as excellent or good by a human literary critic. These results indicate that LLMs such as GPT4 have acquired an emergent ability to interpret complex metaphors, including those embedded in novel poems

The form of analog size information in memory

The information used to choose the larger of two objects from memory was investigated in two experiments that compared the effects of a number of variables on the performance of subjects who either were instructed to use imagery in the comparison task or were not so instructed. Subjects instructed to use imagery could perform the task more quickly if they prepared themselves with an image of one of the objects at its normal size, rather than with an image that was abnormally big or small, or no image at all. Such subjects were also subject to substantial selective interference when asked to simultaneously maintain irrelevant images of digits. In contrast, when subjects were not specifically instructed to use imagery to reach their decisions, an initial image at normal size did not produce significantly faster decisions than no image, or a large or small image congruent with the correct decision. The selective interference created by simultaneously imaging digits was reduced for subjects not told to base their size comparisons on imagery. The difficulty of the size discrimination did not interact significantly with any other variable. The results suggest that subjects, unless specifically instructed to use imagery, can compare the size of objects in memory using information more abstract than visual imagery.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/23015/1/0000584.pd

Emergent Analogical Reasoning in Large Language Models

The recent advent of large language models has reinvigorated debate over whether human cognitive capacities might emerge in such generic models given sufficient training data. Of particular interest is the ability of these models to reason about novel problems zero-shot, without any direct training. In human cognition, this capacity is closely tied to an ability to reason by analogy. Here, we performed a direct comparison between human reasoners and a large language model (the text-davinci-003 variant of GPT-3) on a range of analogical tasks, including a novel text-based matrix reasoning task closely modeled on Raven's Progressive Matrices. We found that GPT-3 displayed a surprisingly strong capacity for abstract pattern induction, matching or even surpassing human capabilities in most settings. Our results indicate that large language models such as GPT-3 have acquired an emergent ability to find zero-shot solutions to a broad range of analogy problems

A positional discriminability model of linearorder judgments

The process of judging the relative order of stimuli in a visual array was investigated in three experiments. In the basic paradigm, a linear array of six colored lines was presented briefly, and subjects decided which of two target lines was the leftmost or rightmost (Experiment 1). The target lines appeared in all possible combinations of serial positions and reaction time (RT) was measured. Distance and semantic congruity effects were obtained, as well as a bowed serial position function. The RT pattern resembled that observed in comparable studies with memorized linear orderings. The serial position function was flattened when the background lines were homogeneously dissimilar to the target lines (Experiment 2). Both a distance effect and bowed serial position functions were obtained when subjects judged which of two target lines was below a black bar cue (Experiment 3). The results favored an analog positional discriminability model over a serial ends-inward scanning model. The positional discriminability model was proposed as a "core model" for the processes involved in judging relative order or magnitude in the domains of memory and perception

Pragmatic reasoning schemas

We propose that people typically reason about realistic situations using neither content-free syntactic inference rules nor representations of specific experiences. Rather, people reason using knowledge structures that we term pragmatic reasoning schemas, which are generalized sets of rules defined in relation to classes of goals. Three experiments examined the impact of a "permission schema" on deductive reasoning. Experiment 1 demonstrated that by evoking the permission schema it is possible to facilitate performance in Wason's selection paradigm for subjects who have had no experience with the specific content of the problems. Experiment 2 showed that a selection problem worded in terms of an abstract permission elicited better performance than one worded in terms of a concrete but arbitrary situation, providing evidence for an abstract permission schema that is free of domain-specific content. Experiment 3 provided evidence that evocation of a permission schema affects not only tasks requiring procedural knowledge, but also a linguistic rephrasing task requiring declarative knowledge. In particular, statements in the form if p then q were rephrased into the form p only if q with greater frequency for permission than for arbitrary statements, and rephrasings of permission statements produced a pattern of introduction of modals (must, can) totally unlike that observed for arbitrary conditional statements. Other pragmatic schemas, such as "causal" and "evidence" schemas can account for both linguistic and reasoning phenomena that alternative hypotheses fail to explain.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/25556/1/0000098.pd

Cognitive reference points in judgments of symbolic magnitude

Research on speeded symbolic magnitude comparisons indicates that decisions are made more quickly when the magnitudes of the stimuli being compared are relatively close to an explicit or implicit reference point. Alternative explanations of this phenomenon are tested by seeking similar effects in nonspeeded rating tasks. In accord with the predictions of discriminability models, rated magnitude differences between stimuli in the vicinity of a reference point are expanded relative to differences between stimuli far from it. The inferred locations of cities along a west-east axis varies systematically depending on which coast, Pacific or Atlantic, is specified as the reference point. Scales derived from the rating data are correlated with the pattern of reaction times obtained in a comparable speeded comparison task. In addition, the distance between the cities nearest the locale of our subjects is subjectively stretched. Reference point effects are also observed when the form of the comparative specifies an implicit reference point at either end of a continuum of subjective size; however, these effects are very small and do not clearly support a discriminability interpretation. Stronger evidence for discriminability effects is obtained when an explicit reference point is established at an arbitrary size value. An implicit scaling model, related to range-frequency theory, is proposed to account for the influence of reference points on relative discriminability of stimulus magnitudes. The implicit scaling model is used to develop an account of how symbolic magnitudes may be learned and of how habitual reference points can produce asymmetries in distance judgments.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/23941/1/0000188.pd

Zero-shot visual reasoning through probabilistic analogical mapping

Human reasoning is grounded in an ability to identify highly abstract commonalities governing superficially dissimilar visual inputs. Recent efforts to develop algorithms with this capacity have largely focused on approaches that require extensive direct training on visual reasoning tasks, and yield limited generalization to problems with novel content. In contrast, a long tradition of research in cognitive science has focused on elucidating the computational principles underlying human analogical reasoning; however, this work has generally relied on manually constructed representations. Here we present visiPAM (visual Probabilistic Analogical Mapping), a model of visual reasoning that synthesizes these two approaches. VisiPAM employs learned representations derived directly from naturalistic visual inputs, coupled with a similarity-based mapping operation derived from cognitive theories of human reasoning. We show that without any direct training, visiPAM outperforms a state-of-the-art deep learning model on an analogical mapping task. In addition, visiPAM closely matches the pattern of human performance on a novel task involving mapping of 3D objects across disparate categories